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Abstract 

This paper introduces a Bayesian framework to detect multiple signals embedded in noisy observations from a sensor array. 
For various states of knowledge on the communication channel and the noise at the receiving sensors, a marginalization procedure 
based on recent tools of finite random matrix theory, in conjunction with the maximum entropy principle, is used to compute the 
hypothesis selection criterion. Quite remarkably, explicit expressions for the Bayesian detector are derived which enable to decide 
on the presence of signal sources in a noisy wireless environment. The proposed Bayesian detector is shown to outperform the 
classical power detector when the noise power is known and provides very good performance for limited knowledge on the noise 
power. Simulations corroborate the theoretical results and quantify the gain achieved using the proposed Bayesian framework. 

I. Introduction 

Since a few years, the idea of smart receiving devices has made its way through the general framework of cognitive radio 
[2]. The general idea of an ideal cognitive receiver is a device that is capable of inferring on any information it is given 
to discover by itself the surrounding environment [12]. Such a device should be first able to turn prior information on the 
transmission channel into a mathematically tractable form. This allows then the terminal to take optimal instantaneous decisions 
in terms of information to feed back, bandwidth to occupy, transmission power to use etc. It should also be capable of updating 
its knowledge to continuously adapt to the dynamics of the environment. This vision of a cognitive radio is compliant with 
Haykin's definition of "brain empowered" wireless devices [12]. 

In particular, one of the key features of cognitive receivers is their ability to sense free spectrum. Indeed, when the cognitive 
device is switched on, its prior knowledge is very limited but still it is requited to decide whether it receives informative data 
or pure noise due to interfering background electromagnetic fields, on different frequency bands: this will be further referred 
to as the signal detection procedure. 

In the single-input single-output antenna (SISO) scenario, the study of Bayesian signal detectors dates back to the work 
of Urkowitz [1] on additive white Gaussian noise (AWGN) channels. It was later extended to more realistic channel models 
[3]-[4]. Urkowitz's signal detector is optimal in the sense that his process performs the maximum correct detection rate, i.e. 
the odds for an informative signal to be detected as such, for a given low false alarm rate, i.e. the odds for a pure noise 
input to be wrongly declared an informative signal. To the authors' knowledge, the multi-antenna (MIMO) extension has not 
been studied, because of the almost prohibitive mathematical complexity of the problem. The usual power detection technique 
■ from Urkowitz was then adapted to the MIMO scenario, e.g. [5]. The latter consists in summing up all the individual powers 
received at each antenna and declaring that the incoming signal carries informatio^ if the total received power exceeds a given 
threshold; otherwise the received signal is declared pure noise. Therefore this technique does not capitalize on the knowledge 
of whatever prior information the receiver might be aware of, apart from an approximate estimation of the signal to noise ratio 
(SNR) required to preset the decision threshold. 

This raises the interest for new techniques such as cooperative spectrum sensing using multiple antennas [5]. Those techniques 
propose to improve the signal detection method of Urkowitz by using extra system dimensions (space dimension through 
cooperation among terminals for example). Unfortunately, the approaches used are highly dependent on the initial assumptions 
made and have led to many different contributions. For instance, some insightful work emerged which uses eigen-spectrum 
or subspace analysis of the received sampled signals [14]-[15]. Those might provide interesting results in their simplicity and 
their limited need for prior system knowledge. However, those studies usually consider static knowledge at the receiver and 
do not cope with the fact that very limited information is indeed provided to the sensing device; this static knowledge often 
includes the a priori exact or approximative knowledge of the SNR0 As an answer to the challenging problem of a priori 
limited channel and noise information, [8] proposes a sub-optimal technique based on the generalized likelihood ratio threshold 
(GLRT); this threshold is shown to merely consist of the ratio between the largest eigenvalue and the trace of the receive 
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empirical covariance matrix and provides a significant improvement compared to the classical detection method based on the 
conditioning number of that matrix. It is also worth mentioning that some contributions treat the problem of estimating the 
number of transmitting sources embedded in noise, e.g. [6], [7], which is more specific than our more general "signal against 
noise" decision criterion; this problem can in fact be treated using the results of this paper, with no need for a complete 
problem redefinition (see Section IVIb . 

In this work, we introduce a general Bayesian framework for signal detection, which is consistent with the receiver's state 
of knowledge on the environment, i.e. which produces a unique transmission model for each prior state of knowledge; this 
knowledge being a list of statistical constraints on the transmission environment. The methodology relies more precisely on 
the work of Jaynes [11] on Bayesian probabilities and especially on the maximum entropy principle. This principle shall allow 
us to provide a consistent detection criterion for each set of prior knowledge at the receiver; here the term consistent must be 
understood in the sense that any alternative method would not be fully compliant (or maximally honest in the own terms of 
Jaynes) with the prior information at the receiver. 

This paper is structured as follows: In Section HI] we introduce the scope and the main results of this paper. In Section Hill 
we formulate the signal detection model. Then in Section |IV] the Bayesian signal detectors are computed for different levels 
of knowledge on the system model. Simulations are then presented in Section [V] Finally, after a short discussion in Section 
IVII on the general framework and its limitations, we provide our conclusions. 

Notations: In the following, boldface lowercase and uppercase characters are used for vectors and matrices, respectively. We 
note {■)^ the Hermitian transpose, tr(-) denotes the matrix trace. 'M,{A, N, M) is the set of matrices of size N x M over the 
algebra A. li(A^) is the set of unitary square matrices of size N. The notation Px{Y) denotes the probability density function 
of the variable X evaluated in the vicinity of Y. The notation {x)+ equals a: if a; > and otherwise. 



II. Main Results 

The main purpose of this work is to propose a universal framework to signal detection, based on cogent information at the 
multi-antenna sensing device. That is, the prior information at the receiver, before channel sensing, will be summarized as a 
set of statistical information about the transmission environment. We will consider here the situation when the additive noise 
power is known perfectly or known to belong to a bounded interval, and the situation when the number of signal sources is 
either known or known to be less than a maximum. 

This information will be translated into its corresponding most appropriate mathematical model, in the sense that this model 
is (i) compliant with the prior information at the receiver and (ii) avoid enforcing empirical (therefore unknown) properties. 
This can be realized thanks to the maximum entropy principle. 

The decision criterion will then be based on the ratio 

CY\im - (1) 

between the hypothesis Jfi of the received data Y containing an information signal and the hypothesis "Kq of its containing only 
noise, when the prior information / (and then its associated maximum entropic model) is considered. Evaluating Cy|/ when 
the system parameters are multi-dimensional could be done by numerical approximation of the underlying integral formulas, 
but then the 'curse of dimensionality' quickly arises and results become very inaccurate, already for little dimensions. This 
obliges one to explicitly compute integrals of (possibly large) matrix parameters, which is performed here thanks to latest 
advances in the field of finite-dimension random matrix theory. 

The main results of this work are summarized in two theorems and some simulation-based observations. First we consider 
the situation when the multi-antenna sensing device is entitled to decide on the presence of a single transmitting signal source 
on a narrow-band channel, assumed static for a given (possibly short) sensing period. This situation is referred to as the SIMO 
scenario. In theorem [T] a closed-form expression of Cy|/ is derived when the maximum entropic model attached to / imposes 
independent Gaussian channel, signal and noise entries; this expression turns out to be solely dependent on the eigenvalues 
of the matrix YY*^, but in a more involved form than the classical power detector. Monte Carlo simulations, supposing an 
accurate evaluation of the transmission model, will in fact show a large detection gain of the novel Bayesian detector compared 
to the classical energy detector. For systems which tolerate very low false alarm decisions, e.g. in a cognitive radio setup when 
secondary users are banned to transmit in bands in use by primary users, the detection gain of our framework are in particular 
observed to be as large as 10% (and possibly more if more numerous simulations are run). 

Theorem |2] generalizes theorem [T] to the scenario when multiple sources are transmitting and their exact number is known to 
the sensing receiver. This is therefore referred to as the MIMO scenario. Simulations reveal less accurate decision capabilities 
in this scenario compared to SIMO. This is interpreted as a consequence of the increased number of variables whose joint 
distribution is less constrained in the MIMO case than in the SIMO case; this makes a specific realization of Y more difficult 
to fit to the underlying MIMO channel model. The MIMO decision is still more efficient than that of the energy detector but 
now the gap between both closes in. 
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Fig. 1. MIMO signal detection configuration 



The most interesting feature of our Bayesian signal detection framework appears when we treat the problem of imprecise 
knowledge about the SNR. In most scientific contributions about signal detection techniques, one assumes perfect or close-to- 
perfect knowledge about the noise power, which is obviously inaccessible to the sensing device, whose objective is precisely 
to separate pure noise from informative signals. We derive in this work a formula for evaluating the presence of an informative 
signal when the knowledge about the noise power may be very limited. It turns out that, even when the a priori noise 
distribution spans uniformly from an (unrealistically) low value to an (unrealistically) large value, simulations results suggest 
that the proposed signal detector performs remarkably well. This is in sharp contrast with classical methods, such as the 
energy detector, subspace-based methods or extreme eigenvalue based methods, for which the information about the noise 
level is required to some extent. This also allows us to go further than in [8], where the GLRT technique was used to cope 
with unknown channel (modelled as a matrix H) and noise information (gathered into the noise power cr^); the decision ratio 
targeted for the GLRT consists in the ratio between supjj ^.2 ^Vci|y,/ and supjj g.2 ^Vci|y,/; this is in particular inaccurate 
when the a priori distribution for the random matrix H is 'broad' around the effective value of the channel matrix. Finally, 
trivial applications of Bayes' rule enable the extension of the current signal detection problem to a wider range of detection 
issues, such as the evaluation of the number of transmitting sources. 



We consider a simple communication system composed of M transmitter sources, e.g. this can either be an A/-antenna 
single transmitter or M single antenna (not necessarily uncorrected) transmitters, and a receiver composed of N sensors, be 
they the (uncorrected) antennas of a single terminal or a mesh of scattered sensors. To enhance the multiple-antenna (MIMO) 
analogy model, the joint set of sources and the joint set of sensors will often be referred to as the transmitter and the receiver, 
respectively. The transmission channel between the transmitter and the receiver is modelled by the matrix H e C^^*^, with 
entries hij, 1 < i < N, 1 < j < M. If, at time I, the transmitter emits data, those are denoted by the Af-dimensional vector 
s*^') = • ■ • , Sm)~^ ^ C^'^ . The additive white Gaussian noise at the receiver is modelled, at time I, as the iV-dimensional 
vector aO^'-^ = o'(6'('', . . . , 6*)^')^ G C^, with the variance of the noise vector entries. For simplicity in the following 
derivations, we shall consider unit variance of the entries of both 6l(') and s('), i.e. £[16(^^12] = 1, E[|sf^|2] = 1. We then 
denote y^'-* = (Vi K ■ ■ ■ lUff y the A^-dimensional data received at time I. Assuming the channel coherence time is at least as 
long as L sampling periods, we finally denote Y — {y^^\ . . . jY*^^^) G c^^^ the matrix of the concatenated receive vectors. 
This scenario is depicted in Figure [U 

Depending on whether the transmitter emits signals, we consider the following hypotheses 

• J-Cq. Only background noise is received. 

• 1 . Informative signals plus background noise are received. 
Therefore, under condition Jio, we have the model. 



III. Signal Model 
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Under this hypothesis, we further denote S the covariance matrix of YY^, 



iL)\ 



(3) 



S = E[YY^] (4) 
= L (HU" + a^Ijv) (5) 
= U (LA) (6) 

where A = diag (ui + a^, . . . ,1/^ + cr^), with {i/j, i € {1,. .., N}} the eigenvalues of HH^ and U a certain unitary matrix. 

The receiver is entitled to decide whether the base station is transmitting informative signals or not; this is, the receiver 
makes a decision over hypothesis JCq or ^i- The receiver is however considered to have very limited information about the 
transmission channel and is in particular not necessarily aware of the exact number M of sources and of the signal-to-noise 
ratio. For this reason, the maximum entropy principle requires that all unknown variables be assigned a probability distribution 
which is both (i) consistent with the prior information (voluntarily discarding information violates the Bayesian philosophy) 
and (ii) has maximal entropy over the set of densities that validate (i). It is known in particular that, upon the unique constraint 
of its covariance matrix, the entropy maximizing joint distribution of a given vector is central Gaussian. Therefore, if H is 
only known to satisfy, as is often the case in the short term, E[trHH'^] = 1, the maximum entropy principle states that the 
hij 's are independent and distributed as hij ~ C?s[(0, l/A{f). For the same reason, both noise 0^ ^ and signal ^ variables are 
taken as independent central Gaussian with variance 

The decision criterion for the receiver to establish whether an informative signal was transmitted is based on the ratio C, 

c(y) = p^^'^i^i (7) 

Thanks to Bayes' rule, this is 

ciY) = f";:"'"-^ (8) 

with Pj{. the a priori probability for hypothesis IKj to be true. We suppose that no side information allows the receiver to 
think CKi is more or less probable than IKq, and therefore set PjCi = P^o — 5 (this again validates the maximum entropy 
principle), and then 

reduces to a maximum likelihood criterion. 

In the next section, we will derive close-form expressions for C(Y) under the hypotheses that the values of M and the 
SNR are either perfectly or only partially known at the receiver. 



IV. Signal Detection 

A. Known noise variance and number of signal sources 

1) Derivation of pY\:Ki in SIMO case: We first analyze the situation when the noise power cr^ and the number M of signal 
sources are known to the receiver. We also assume in this first scenario that M = 1. Further consider that L > N, which is a 
common assumption. 
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a) Pure noise likelihood Py\:Ko- this first scenario, the noise entries are Gaussian and independent. The distribution 
for Y, that can be seen as a random vector with NL entries, is then a NL muhivariate uncorrected complex Gaussian with 
CO variance matrix a^l^L, 



1 -Jrj-trYY^ 
p ^ 

(11) 

by denoting x = {xi, . . . , the eigenvalues of YY^, ( fT2]) only depends on J2iLi 



b) Informative signal likelihood -Py|Mi-' In scenario !Ki, the problem is more involved. The entries of the channel matrix 
H were modelled as jointly uncorrelated Gaussian distributed, with E[|/iijp] = 1/M. Therefore, since M = 1, H E C^^^ 
and S = HH*^ +a^lN has iV— 1 eigenvalues equal to and another distinct eigenvalue Xi = vi + cr^ — (X^iLi l^iiP) + cr^- 
The density of Ai — cr^ is a complex Xn distribution (which is, up to a scaling factor 2, equivalent to a real X2n distribution). 
Hence the eigenvalue distribution of S, defined on M+^, 



1 p-(Ai-'t') JL 

P,(A) =-(A, - n ^(A. - a^) (13) 

Given model Y is distributed as correlated Gaussian, 



PY|i:,/,(Y,S)--^^^^-^e (14) 

where /j. denotes the prior information and M — fc". 

Since the channel H is unknown, we need to integrate out all possible channels of the model ^ over the probability space 
of X M matrices with Gaussian i.i.d. distribution. This is equivalent to integrating out all possible covariance matrices S 
over the space of such positive definite Hermitian matrices 

iV|Mi(Y)-/ PY|SMi(Y,S)Ps(S)dS (15) 

In the following, we shall prove that the integral ( fTsT i can be derived as the following closed-form expression. 
Theorem 1: The detection ratio Cy|/i(Y) for the presence of an informative signal under prior information Ii, i.e. the 
receiver knows (i) M = 1 signal source, (ii) the SNR cr^^, reads 

1 g.2(W+L-l) ^T^ + f^ 

^m(Y) -jrE-Fpr-, -Jn-l-i{<7',x,) (16) 

1=1 

with xi, . . . , xjv the empirical eigenvalues of YY'^ and where 

Jk{x,y) = / t'^e^'-Tdt (17) 

Proof: We start by noticing that H is Gaussian and therefore the joint density of its entries is invariant by left and right 
unitary products. As a consequence, the distribution of the matrix S = HH'^ + cr^I is unitarily invariant, i.e. for any unitary 
matrix V, VSV'^ has the same joint density as S. The latter density does not as a consequence depend on U in its singular 
value decomposition This allows us to write, similarly as in [10], 

^YiMi(Y) =^PY|5:,Mi(Y,S)Ps(S)dS (18) 
= / PY|s,JCi(Y,S)FiA(iA)dUd(iA) (19) 

JU(Af)xR+" 

= / PY|i:.Mi(Y,S)PA,(Ai)dUdAi (20) 

JU(Af)xR+ 
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Equation ( [20] | leads then to 



/U(A')x 

To go further, we utilize the Harish-Chandra identity [17] 

/N-i \ det ( {e"^'^-'}i<i<Ar I 

7uw Vnii / A(A)A(B) 

in which, for a matrix X with eigenvalues xi, . . . ,X]y, A(X) indicates the Vandermonde determinant 

A(X) - J](x,-x,) (23) 

And then Pyi/i(Y) further develops as 



jv(jv-i) .| „+^, , det |e [i<,,<jv 



A2..."a«^'t^ 7r^^a2-E'(^-i)iV! 7^2 ^ ^ A(X)A(A) 



p-'n^-S-l /•+°° 1 det,^. '/i<z<w 



= , / Tx(Ai~^-r-^e--Met(A-v-0 \.^A..?r'^ ^A, (25) 



n'^-Sl /■+- detNe '^ } i< 



in which X and xi, . . . ,xn respectively correspond to YY'^ and its eigenvalues. The equality ( l25T l comes from the fact that 

A(A-l) - ( l)N(N+3)/2 A(A) 

By denoting y = (y^, . . . , y^^i, vn) ^ {^2, ■ ■ ■ , >^n , Ai) and the functions, 

/(a;„y,)=e"^ (27) 
= /(a^^,yj) (28) 

we can perform a similar derivation as in [13] to obtain 



det 



A2,...,A„^<t2 A(X)A(A) yu-,yN-l^'r^ A(X)A(A) 

j/jv— >Ai 



det[/.(a^), //(g^),..., /<^~^H^^), /.(Ai)] 
n.<,(^.-^.)(Ai-'T2)^-inf=f j! 



(30) 



The change of variables led to a switch of one column and explains the (—1)^^^ factor when computing the resulting 
determinant. 

The partial derivatives of / along the second variable is 

Af) («^^)=E^^r)^«-e-t (31) 

Jk>i 

= Kk{a,b)e^'^ (32) 
Back to the full expression of Py|Mi(Y), we then have 
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^Y|/,(Y) 



g<T=^2(JV-l)(JV-L-l) + C 



^a'^^2(N-l)(N-L-l) p + oo 



dXi (33) 



I e -^1 



<i<N I 
l<j<Af-2 I 



e -^1 



(34) 



Before going further, we need the following result, demonstrated in Appendix H] 
Lemma 1: Given a family {ai, . . . , OAr} £ M^, N > 2, and 6 G M*, we have 



dot 



1 



{Kj{ai,b)) i<i<N 



l<j<N-l 



N-1) n^^j' 



(35) 



1<J 



By factorizing every row of the matrix by e ^ and developing the determinant on the last column, one obtains 

gO-=^^2(JV-l)(Ar-L-l) /•+00 



1 V-N 



■r^JV — "■^/l "^^ — ~~ — 2" I 



■(-l)^-i^(-l) 



dXi 



2 1 s-^N 



^l)(L-l) XI' 



Ar_LAr^2(7V-l)(L-l) Zl^ T-rAf / \ „ 



which finally gives 



2 1 x^N 



^Y|/,(Y) = 



N 



with 



(37) 

(38) 
(39) 

(40) 



+ 00 



2y— K^k-i{2Vy) - ee-'-^dt 
Jo 



where Kn denotes the modified Bessel function of the second kind. 
We finally have the desired decision criterion 



N 



-JN-L-lio"^ ,Xl) 



(41) 

(42) 



(43) 



2) Derivation of PY\3<i in MIMO case: In the MIMO configuration, pY\yLo remains unchanged and equation ( fT2l l is still 
correct. For the subsequent derivations, we only treat the situation when M < N but the case M > iV is a trivial extension. 

In this scenario, H £ ^JVxm already mentioned, distributed as a Gaussian i.i.d. matrix. The mean variance of every 

row IS 

EE?li = 1- Therefore AfHH*^ is distributed as a standard Wishart matrix. Hence, observing that S — cr^I 

■' LJ 

is the diagonal matrix of eigenvalues of HH , 



N 



E = U • diag(z^i + cr^, . 
the eigenvalue distribution density of A can be derived [9] 



^2 2 2\ ttH 



(44) 
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^a(A) 



(N ~ M)\M^ 



n 



(\ 9\N-M M N 



(45) 



1=1 ^ ' '' i<j i>M 

From the equations (l44l l and ( |45] ) above, we will now show the MIMO equivalent result to theorem [U which unfolds as 
follows, 

Theorem 2: The detection ratio Cy|/„ (Y) for the presence of informative signal under prior information Im, i-S- when the 
receiver is aware of (i) M < N signal sources, (ii) the SNR cr^^, reads 



M 



^^•^"^ llj=i ac[i,JV] 11 11 l^ai beg'(M) ;=i 



(46) 



with CP(A'/) the ensemble of permutations of {1, . . . , Af }, b = (61, . . . , 6a/) and sgn(b) the signature of the permutation b. 



Proof: We note first that for any couple ( Aj , \j ) of the M largest eigenvalues of S, j 7^ i, (-^1 1 ■ ■ • j -^s 



. A,- 



Pa(Ai, . . . , Aj 

in the case M > 1, those two conditions are sufficient [10] to ensure 



, . . . , Xj, . . . , Xm 



, Xm), where A is the joint (unordered) random variable (Ai, . . . , Am)- Since H is still isometric 



Py|/..(Y) = £pY|i:j„(Y,S)Ps(S)dS 

PY|i:,/„(Y,S)PA(A)rfUdA 



/U(Ar)xR+™ 

which, using the same technique as previously, is further developed into 



(47) 
(48) 



Py|/„(Y)= lim 



^ ^ i—1 i<j 



det 



<i<7V 
l<j<7V^ 



A(X)A(A) 



-dXi . . . dX 



M 



(49) 



+ 00 p-\-oc 



M 



Ha, 



Y[i<j i^i ~ ^3) 



det 



e 



e ^1 



e ^1 
(50) 



in which the term (—1)^"™ 2 originates from the M exchanges between the fc column and the \N ~ k + 1) column, 

fce[l,M]. 

By factorizing the determinant by ^i=i developing along the M last columns, we have from Lemma [T] 
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det 



e ^ 



Kj{xi,a'^)e ^ 



^1 _ ^1 



e ■ ■ ■ e 



(51) 



:e-%^ ^ (_l)[^+ai]e-^»i(^-^^)... 

aie[l,JV] 
AT 

X ^ (•_]^')[jV-M+l+OM-Ei<M^(««<"M)] 



^2(Ar-M-l)(iV-M) il ~ 



aC[l,JV] 



n (^'-^j) 



= e ^ 



Together, this becomes, 



(52) 
(53) 

(54) 
(55) 



Y|/m( ) - j^^_^NL^2{N-M)(L-M) ]J^_-^ j\ 



J-^^ J<^^ i=l i<j aC[l,JV] ^J'' 



{N -M)\e" " ^ 



^\M{M-1L-\)MI1^NL^2{N-M){L-M) j\ 



X > „ „ -. r / ••• / e -'=^v' A, y I I A," M I ( A, - A, IdAi . . . c^Am 

(57) 

(58) 

i<j bey(M) i=l 

where T'(fc) is the ensemble of permutations of k and sgn(b) designs the signature of the permutation b. Recognizing the 
expression of J^, we finally obtain 

^Y|/„(Y) = 



Remind now the Vandermonde determinant identity 

M M 



]^\^NLfj2(N-M){L-M) f]^"! j\ 



M 



IV 1 

[i,N] 11 11 (^"i a;j) bey(M) 



(59) 



1=1 



For instance, when M = 2, we have 
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(60) 

in which jjf'^ = Jk{2a'^,2x). 

Decisions regarding the signal detection are then carried out by computing the ratio CY|/j(f (Y) between equation ( [59] ) and 
equation ST% as follows 



y^I^ I 

' iv!Af (M-i--)M/2 n-- ,1 ^^^^^ n n - b.irM) 

(61) 
■ 

Note that Cy|/j,j (Y) is a function of the empirical eigenvalues xi, . . . ,xn of YY'^ only. This is explained by the presence of 
only Gaussian random entities, whose isometric property leads the eigenvectors of YY*^ to contain no additional information. 
Remark also that equation ( |46] l is a non-trivial function of xi, . . . , xn, which does not involve the sum of the x^'s as for the 
case of the classical energy detector 

In the following, we extend the current signal detector to the situations where M and are not a priori known at the 
receiver. 



B. Number of sources and/or noise variance unknown 

1) Unknown noise variance: Efficient signal detection when the noise level is unknown is highly desirable. Indeed, if the 
noise level were exactly known, some prior noise detection mechanism would be required. The difficulty here is handily 
avoided thanks to ad-hoc methods that are asymptotically independent of the noise level [14]-[15]. Instead, we shall consider 
some prior information about the noise level. Establishing prior information of variables defined in a continuum is still a 
controverted debate of the maximum entropy theory. However, a few solutions are classically considered that are based on 
desirable properties. Those are successively detailed in the following. 

Two classical cases are usually encountered, 

• the noise level is known to belong to a continuum [cr^, u^]. If no more information is known, then it is desirable to take 
a uniform prior for and then 

P^2{a^)dcr^ = -^-^—^da^ (62) 

However, a questionable issue of invariance to variable change arises. Indeed, if P^^ is uniform, the distribution associated 
to the variable a = Va^ is then non-uniform. This old problem is partially answered by Jeffreys [16] who suggests that 
an uninformative prior should be any distribution that does not add information to the posterior distribution Pa2\-Y.ij,i 
(for recent developments, see also [18]). However, in our problem, the uninformative prior is rather involved so we only 
consider uniform prior distribution (l62T i for (we denote I'j^j = ct^ e [ct^, it^]") and therefore 



P^K, = ^r^_ jj PYi.^J'JY,a')da' (63) 

• one has no information concerning the noise power. The only information about is > 0. Again, we might want to 
subjugate <t^ to Jeffreys' uninformative prior. However, computing this prior is again rather involved. The other alternative 
is to take the limit of ( |63] l when tends to zero and tends to infinity. This Umiting process produces an improper 
integral form. This would be, with I'j^j the updated background information, 

Py|/;', = lim I PY\a-{Y,a^)da^ (64) 

This leads in any case to the updated decisions Cy\m of the form, 



11 



Bayesian Detector Power Detector 




-20 20 40 60 80 100 120 140 0.5 1 1.5 2 2.5 3 



C(Y) [dB] C(Y) 

Fig. 2. Power detection performance in SIMO - M = 1, A'^ = 4, L = 8, SNR = —3 dB. On the left, Bayesian detector; on tlie right, classical power 
detector. 



Cy|m = —2 (65) 

The computational difficulty raised by the integrals Jk{x, y) does not allow for any satisfying closed-form formulas for (1631) 
and ( l64b . In the following, we therefore only consider the bounded continuum scenario. 



C. Unknown number of sources M 

In practical cases, the number of transmitting sources is only known to be finite. If only an upper bound value A/„iax on M 
is known, a uniform prior is assigned to M (which is again compliant with the maximum entropy principle). The probability 
distribution of Y under hypothesis /q ="(t^ known, M unknown", reads 



F(Y|/o) = Pi^V'M = ^",/o) • -P("M = z"|/o) (66) 

i=l 

= E ^(Y|"A/-z",/o) (67) 



which does not meet any computational difficulty. 
This leads then to the decision ratio CyIct^, 



^-^YI(t2 — T7 (OO) 



V. Simulation and Results 

In the following, we present results obtained for the aforementioned SIMO and MIMO scenarios, using theorems [T| and |2] 
respectively. In the simulations, the hypothesis concerning incoming data, channel aspect and noise figure are those presented 
in the model of Section HIlFI i.e. the channel, signal and noise matrix entries are i.i.d. Gaussian with respective variance 1/M, 
1 and 1. The results are validated by and compared with the classical power detector, which merely consists in summing the 
eigenvalues xi,. . .,xn of YY'^ and assumes an empirical decision threshold; the corresponding decision is then based on 
the scaled value 



'it might be objected that assuming the maximum entropy distributions in the simulations is dishonest since the real distributions are unknown; the 
simulations here however intend only to verify the validity of our theoretical results and cannot be used as any proof of performance. 
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Fig. 3. CDR against FAR for SIMO transmission - M 
practical interest. 



1, Af = 4, L 



SNR 



-3 dB. On the left, full FAR range; on the right, FAR range of 




Fig. 4. Correct detection rates under FAR constraints for different SNR levels, M = 1, N = A, L = 8 




Fig. 5. CDR against FAR for SIMO transmission - M = 2, Af = 4, L = 8, SNR = 



—3 dB. FAR range of practical interest. 
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C{Y) 



LNcT^ 




(69) 



In our first example, we consider a SIMO channel with TV = 4 antennas at the receiver, L — 8 sampling periods and a 
signal to noise ratio SNR = —3 dB. For fair comparison with classical signal detection algorithms, we stick to the false alarm 
rate (FAR) against correct detection rate (CDR) performance evaluation. Figure |2] presents the respective FAR and CDR for 
the novel Bayesian estimator and for the classical power detector, obtained on 100, 000 Monte Carlo channel realizations. 
Depending on the acceptable FAR, the decision threshold for the power detector is somewhere around 1 while the threshold 
for the Bayesian approach is somewhere around C(Y) = dB. Since both algorithms scale very differently, fair comparison is 
obtained by plotting the CDR versus FAR curve. This is depicted in Figure [3] For large values of the FAR, both Bayesian and 
power detectors produce similar performance. In most practical applications, one is however interested into high performance 
CDR for fixed low FAR (in the order of 10^'^ to 10^^ depending on the application); for this range of FAR, it is observed, 
in the right part of Figure [51 that as much as a 10% increase in detection ability is obtained by the Bayesian detector and 
this gain increases along with smaller FAR. This is confirmed by Figure |4] in which the performance of the Bayesian signal 
detector with respect to the energy detector for different constrained FAR is presented against the SNR. 

In Figure |5] we took iV = 4, L = 8 and SNR = —3 dB as before but consider now M — 2 signal sources; we then use 
theorem |2] here. In this scenario the classical power detector closes in the gap with the Bayesian detector, compared to the 
SIMO situation. Having depicted in the same plot the detection performance for M ~ 1, we notice that the classical energy 
detector performs better in the scenario M = 2, which might be interpreted as a result of a channel hardening effect [19]; on 
the contrary, the Bayesian detector performs less accurately in this case, which can be attributed to the increased number of 
random variables (both in the channel matrix H and in the signal matrix 0) inducing increased 'freeness' in the interpretation 
of the hypothetical origins of the output matrix Y. 

Consider now the scenario when the noise variance cr^ is a priori known to belong to the interval [crijCr^]. The two- 
dimensional integration of equation ( |62] | is prohibitive for producing numerical results. We therefore divide the continuum 
[(t^_,ct\] into K subsets [a^_ + k/\{a'^),a^_ + (k + l)A(cr2)], for A: e {0, . . . , if - 1} and A(cr2) = - cr'^_)/K where 
A((T^) is chosen small enough so to produce a rather good approximation of ( l62b . This is presented in Figure |6] which 
demonstrates the effect of an inaccurate knowledge of the noise power in terms of CDR and FAR. In this simulation, M = 1, 
N = A, L = % and SNR = dB. Comparison is made between the cases of exact SNR knowledge, short SNR range 
[(t'^_,cj\] = [-2.5, 2.5] dB discretized as a set {-2.5, -1.5, . . . , 1.5, 2.5} dB, large SNR range [cr'l.aW = [-5, 5] dB discretized 
as a set {—5, —4, . . . , 4, 5} dB and very large range [cr^, ct^] = [—9, 9] dB discretized as a set {—9, —8, . . . , 8, 9} dB. Observe 
that the short SNR range provides already a strong performance decay compared to the ideal scenario, which is particularly 
noticeable in terms of CDR performance at low FAR. Larger SNR ranges are then only slightly worse than the short range 
scenario and seem to converge to a 'worst-case limit' this can be interpreted by the fact that the additional hypotheses, i.e. 
very strong or very little noise power, are automatically discarded as the values of Py|(t2,/{ (Y, cr^) and Py|(t2,Mo(Y, cr^) 
become negUgible for unrealistic values of cr^. Additional simulations for larger SNR ranges were carried out that visually 
confirm that the FAR and CDR plots are identical here as long as cr'i < — 5 dB and ct^ > 5 dB. Therefore, simulations suggest 
that the proposed Bayesian signal detector is able to cope even with totally unknown SNR, which is obviously not the case of 
the classical energy detector that relies on an SNR-dependent decision threshold. 



In the previous framework, we relied on the maximum entropy principle in order to derive unique a priori distributions for the 
various unknown system parameters. The provided Bayesian solutions, derived from the channel state of knowledge available 
at the receiver, were claimed consistent in the proposed probability framework. This framework is in particular extensible to 
whatever prior knowledge the receiver might have on the transmission environment. However, some limitations can be raised. 
First, as stated in IIV-B.U uninformative priors modeling is still an incomplete and controversial theory, for which no definite 
answer is available to this day. When such a prior information is to be treated, the proposed signal detection framework is not 
capable of singling out a proper maximum entropic model; this constitutes a major coherency issue of our source detection 
framework. Also, the mathematical tools to derive maximum entropy distributions, e.g. Lagrangian multipliers, only cope with 
statistical prior knowledge, such as the moments of the underlying density functions, and are rarely able to treat deterministic 
knowledge. 

Note however that the advances in the field of random matrix theory provides new answers to problems of high dimensionality, 
even for finite N,L < oo values. Those problems, such as the present maximum-likelihood multi-antenna signal detection, are 
often considered intractable and suffer in practice from the so-called curse of dimensionality. The current study relies nonetheless 
on the important property that the transmission channel H is modelled as i.i.d. Gaussian; if H were more structured, it would 



VI. Discussion 



*note that the plot for the very large SNR range was intentionally removed from the left hand-side plot for it almost perfectly fitted to plot corresponding 
to the large SNR range. 
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Fig. 6. FAR and CDR, for different a priori information: exact SNR (0 dB), sliort range SNR ([-2.5, 2.5] dB) and lai'ge range SNR ([-5, 5] dB) SNR, 
M = 1, N = i, L = 8, true SNR = dB 



have been more difficult to obtain an integral expression similar to ( l48T l and the final results, be it derivable, would implicate 
not only the eigenvalues but also the eigenvectors of YY^. 

]VIore importantly, the proposed Bayesian framework allows one to answer a wider scope of problems than the present 
multi-source signal detection. In particular, we discussed in the introduction the somewhat different problem of counting the 
number of transmitting sources, which has received a lot of interest. In our framework, this consists in considering a set of 
hypotheses J{q, . . . , "^'m^^^^ where is the hypothesis "fc signals are being transmitted" for a maximum of Afmax sources, 
and evaluating the M^ax hypothesis tests 



Cfe|Y(Y) 



WAY 



(Y) 



iV|"M=fe"(Y)P"M=-t 



E;^r^^iY(Y) 

which can be evaluated using the results derived above. 



'(Y)P"M=. 



(70) 



VII. Conclusion 

In this work, we introduced a general Bayesian framework for multi-antenna multi-source detection, which can be used 
to detect a single multi-antenna source emitter as well as multiple single antenna transmitters. This framework is based on 
a consistent treatment of the system information available at the sensing device. The performance of the novel Bayesian 
multi-signal source detector is derived and compared in simulations with the classical power detection technique. We observed 
in particular that the proposed Bayesian detector performs better than the classical energy detector for low false alarm rate 
constraints, and is capable of treating the problem of signal detection under imprecise or completely unknown signal-to-noise 
ratio. 

Appendix I 
Proof of Lemma[T] 

We will show the result by recursion. For N = 2, the determinant is simply (02 — aij/b"^ which is compliant with the 
formula. Assuming the result for — 1, we develop the determinant for dimension N on the last column (that corresponding 

to j = A^ — 1), to obtain 



N 



^ m!(A^-l-m)!(m-l)!6^-i+™ ' b{N-2){N-i) 1 

-1 / V / ^ 



:)(Af-l) II 



(71) 



<6 



which, after development of the Vandermonde determinant leads to 



X^^^l^N+^ (_i)jv i+'"(A^- l)!(Af-2)! ^ sgnfCT-)r(^-2)(A'-i) TT , 

i=l m=l ^ J \ J cr.gg).(jv_i) k=l 



,(fc)-l 



(72) 



15 



where 'J'i{N — 1) is the set of all functions that map {1, . . . + . . . , N} onto {1, . . . , iV — 1} (this can be thought of as 

the set of permutations of N~ 1 with i in input labelled by N). Now notice that in the expression above, for 1 < m < one 
can exchange and a;^''*^' ^ for the unique k such that ai{k) — I — m. However, this comes along with a change of the term 
(— 1)^+' by (—1)^+'^' (hence a multiplication by (— 1)*^^*) and a change of the signature of the 'pseudo-permutation' a.i as 
sgn(crj)(— l)*"'^*^^ (k — i — 1 being the number of steps needed to bring k before i + 1 by successive neighbor-permutations); the 
rest of the expression is not affected. Therefore, those two instances sum up to in the above expressions, for any m < N — 2. 
This leads to consider only the term — 1 in the sum X]m=i- This reduces the above expression to 

N N N N 

i=l a,e-P,{N-l) k=l i=l tTieVdN) k=l 

k^i 

with Vi{N) the set of permutations ct of for which f7(i) ~ N (which imposes a product by (—1)^^* to the previous 
signature and the collapse of the remaining (—1)^+* term). Now, 7{N), the set of all permutations of N satisfies 

'P{N) = y,{N)U{a (zy{N)\a{i) ^ N} (74) 

N-l 

= %{N) U\J {ae nN)\a{t) - k} (75) 
the right-hand side member of the above equation is obtained by all the elements in the sum of (l73T l. Hence the final result. 
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